The last few years in imaging technology has witnessed an advent of ubiquitous high-definition (HD) image display systems, as well as an exponential growth in the number of low-resolution image acquisition devices, such as cellular telephones, laptops, web cameras and other hand-held devices.
Given this scenario, the classical problem of image upscaling assumes a renewed significance. Image upscaling, also variously termed as upscaling and super-resolution, is a highly ill-posed linear inverse problem, because the number of unknowns, i.e., high-resolution image pixel values, exceeds the number of observations, i.e., low-resolution pixel values, by an order of magnitude, even at moderate upscaling factors.
The challenge is exacerbated by the unique difficulties encountered in modeling images of natural scenes, as well as the inevitable presence of added nuisances such as camera blur and noise in captured images.
Image upscaling methods can be catecorigized as parametric and non-parametric. Methods belonging to the first category assume parametric models for the output high-resolution image. The prominent method in this class uses bicubic interpolation. That method assumes a bandlimited structure in images. The method is indeed the most common in commercial photo editing packages such as Adobe Photoshop.
Over the last ten years, more sophisticated parametric models have been developed. Total variation minimization methods assume that images have bounded total variation (TV) norms. Probabilistic models assume a prior on gradient profiles of images, and specify or estimate from data the hyperparameters of these priors. Sparse models assume that image patches are sparse in a basis or learned dictionary.
Methods belonging to the second category do not assume an explicit model for images or image patches; instead those methods exploit natural image invariances, such as scale- or translation-invariance. In such cases, the prior for the high-resolution unknown image patches are raw patches from a database of external images, or even patches from the scale-space input image. Those methods “learn” correspondences between high-resolution and low resolution patches and store the correspondences in a searchable database. During reconstruction, every occurrence of a low-resolution patch is replaced by its corresponding high-resolution patch.
Both categories typically have one or more of the following drawbacks.
Loss of Fine Details:
Neither category of methods seems to reproduce realistic textural details for even moderate upscaling factors.
Visual Artifacts:
The resulting images suffer from various artifacts such as blurry edges, exaggerated noise, halos, ringing and aliasing artifacts, (staircases or jaggies).
Computational Demands:
All prior art methods involve considerable computational costs. Upscaling a small (128×128) image by a factor of four might typically take several minutes.